Social touch in human–computer interaction

Touch is our primary non-verbal communication channel for conveying intimate emotions and as such essential for our physical and emotional wellbeing. In our digital age, human social interaction is often mediated. However, even though there is increasing evidence that mediated touch affords affective communication, current communication systems (such as videoconferencing) still do not support communication through the sense of touch. As a result, mediated communication does not provide the intense affective experience of co-located communication. The need for ICT mediated or generated touch as an intuitive way of social communication is even further emphasized by the growing interest in the use of touch-enabled agents and robots for healthcare, teaching, and telepresence applications. Here, we review the important role of social touch in our daily life and the available evidence that affective touch can be mediated reliably between humans and between humans and digital agents. We base our observations on evidence from psychology, computer science, sociology, and neuroscience with focus on the first two. Our review shows that mediated affective touch can modulate physiological responses, increase trust and affection, help to establish bonds between humans and avatars or robots, and initiate pro-social behavior. We argue that ICT mediated or generated social touch can (a) intensify the perceived social presence of remote communication partners and (b) enable computer systems to more effectively convey affective information. However, this research field on the crossroads of ICT and psychology is still embryonic and we identify several topics that can help to mature the field in the following areas: establishing an overarching theoretical framework, employing better research methodologies, developing basic social touch building blocks, and solving specific ICT challenges

Effects of mediated social touch on affective experiences and trust

This study investigated whether communication via mediated hand pressure during a remotely shared experience (watching an amusing video) can (1) enhance recovery from sadness, (2) enhance the affective quality of the experience, and (3) increase trust towards the communication partner. Thereto participants first watched a sad movie clip to elicit sadness, followed by a funny one to stimulate recovery from sadness. While watching the funny clip they signaled a hypothetical fellow participant every time they felt amused. In the experimental condition the participants responded by pressing a hand-held two-way mediated touch device (a Frebble), which also provided haptic feedback via simulated hand squeezes. In the control condition they responded by pressing a button and they received abstract visual feedback. Objective (heart rate, galvanic skin conductance, number and duration of joystick or Frebble presses) and subjective (questionnaires) data were collected to assess the emotional reactions of the participants. The subjective measurements confirmed that the sad movie successfully induced sadness while the funny movie indeed evoked more positive feelings. Although their ranking agreed with the subjective measurements, the physiological measurements confirmed this conclusion only for the funny movie. The results show that recovery from movie induced sadness, the affective experience of the amusing movie, and trust towards the communication partner did not differ between both experimental conditions. Hence, feedback via mediated hand touching did not enhance either of these factors compared to visual feedback. Further analysis of the data showed that participants scoring low on Extraversion (i.e., persons that are more introvert) or low on Touch Receptivity (i.e., persons who do not like to be touched by others) felt better understood by their communication partner when receiving mediated touch feedback instead of visual feedback, while the opposite was found for participants scoring high on these factors. The implications of these results for further research are discussed, and some suggestions for follow-up experiments are presented

Tactile Roughness Perception in the Presence of Olfactory and Trigeminal Stimulants

Previous research has shown that odorants consistently evoke associations with textures and their tactile properties like smoothness and roughness. Also, it has been observed that olfaction can modulate tactile perception. We therefore hypothesized that tactile roughness perception may be biased towards the somatosensory connotation of an ambient odorant. We performed two experiments to test this hypothesis. In the first experiment, we investigated the influence of ambient chemosensory stimuli with different roughness connotations on tactile roughness perception. In addition to a pleasant odor with a connotation of softness (PEA), we also included a trigeminal stimulant with a rough, sharp or prickly connotation (Ethanol). We expected that - compared to a No - odorant control condition—tactile texture perception would be biased towards smoothness in the presence of PEA and towards roughness in the presence of Ethanol. However, our results show no significant interaction between chemosensory stimulation and perceived tactile surface roughness. It could be argued that ambient odors may be less effective in stimulating crossmodal associations, since they are by definition extraneous to the tactile stimuli. In an attempt to optimize the conditions for sensory integration, we therefore performed a second experiment in which the olfactory and tactile stimuli were presented in synchrony and in close spatial proximity. In addition, we included pleasant (Lemon) and unpleasant (Indole) odorants that are known to have the ability to affect tactile perception. We expected that tactile stimuli would be perceived as less rough when simultaneously presented with Lemon or PEA (both associated with softness) than when presented with Ethanol or Indole (odors that can be associated with roughness). Again, we found no significant main effect of chemosensory condition on perceived tactile roughness. We discuss the limitations of this study and we present suggestions for future research.Previous research has shown that odorants consistently evoke associations with textures and their tactile properties like smoothness and roughness. Also, it has been observed that olfaction can modulate tactile perception. We therefore hypothesized that tactile roughness perception may be biased towards the somatosensory connotation of an ambient odorant. We performed two experiments to test this hypothesis. In the first experiment, we investigated the influence of ambient chemosensory stimuli with different roughness connotations on tactile roughness perception. In addition to a pleasant odor with a connotation of softness (PEA), we also included a trigeminal stimulant with a rough, sharp or prickly connotation (Ethanol). We expected that—compared to a No-odorant control condition—tactile texture perception would be biased towards smoothness in the presence of PEA and towards roughness in the presence of Ethanol. However, our results show no significant interaction between chemosensory stimulation and perceived tactile surface roughness. It could be argued that ambient odors may be less effective in stimulating crossmodal associations, since they are by definition extraneous to the tactile stimuli. In an attempt to optimize the conditions for sensory integration, we therefore performed a second experiment in which the olfactory and tactile stimuli were presented in synchrony and in close spatial proximity. In addition, we included pleasant (Lemon) and unpleasant (Indole) odorants that are known to have the ability to affect tactile perception. We expected that tactile stimuli would be perceived as less rough when simultaneously presented with Lemon or PEA (both associated with softness) than when presented with Ethanol or Indole (odors that can be associated with roughness). Again, we found no significant main effect of chemosensory condition on perceived tactile roughness. We discuss the limitations of this study and we present suggestions for future research

Measuring dashboard performance.:Optimizing the view on data

Due to the recent technological advancements in data collection, transmission and storage, the amount of data that is available in private or publically accessible databases is growing exponentially. In principle this data may enable individuals and organizations to make well-informed decisions and timely adapt to changing conditions. However, as datasets increase in size and complexity, it becomes more and more difficult to explore the data, select the relevant information, perceive patterns and interpret the data correctly to make the right decisions. Efficient and effective information visualization tools that allow a user to explore and understand the data in an intuitive manner may serve to achieve this goal. Dashboards are promising candidates for this purpose. Dashboards are graphical user interfaces consisting of different components, that organize and present information in a way that is supposedly easy to read and comprehend. The overall quality of dashboards depends on the quality of their components and the synergy between them. Because of their inherent complexity, determining the overall quality of dashboards is difficult. We are currently developing a framework to evaluate and optimize the performance of dashboards. Such a framework will enable the design of efficient and effective dashboards that provide users with an intuitive view on data. www.humanfactors.n

Emotional Responses to Multisensory Environmental Stimuli: A Conceptual Framework and Literature Review.

How we perceive our environment affects the way we feel and behave. The impressions of our ambient environment are influenced by its entire spectrum of physical characteristics (e.g., luminosity, sound, scents, temperature) in a dynamic and interactive way. The ability to manipulate the sensory aspects of an environment such that people feel comfortable or exhibit a desired behavior is gaining interest and social relevance. Although much is known about the sensory effects of individual environmental characteristics, their combined effects are not a priori evident due to a wide range of non-linear interactions in the processing of sensory cues. As a result, it is currently not known how different environmental characteristics should be combined to effectively induce desired emotional and behavioral effects. To gain more insight into this matter, we performed a literature review on the emotional effects of multisensory stimulation. Although we found some interesting mechanisms, the outcome also reveals that empirical evidence is still scarce and haphazard. To stimulate further discussion and research, we propose a conceptual framework that describes how environmental interventions are likely to affect human emotional responses. This framework leads to some critical research questions that suggest opportunities for further investigation

Sensators: active multisensory tangible user interfaces

Although Tangible User Interfaces are considered an intuitive means of human-computer interaction, they oftentimes lack the option to provide active feedback. We developed ‘Sensators’: generic shaped active tangibles to be used on a multi-touch table. Sensators can represent digital information by means of ‘Sensicons’: multimodal messages consisting of visual, auditory, and vibro- tactile cues. In our demonstration, we will present Sensators as suitable tools for research on multimodal perception in different tangible HCI tasks